Why you may predict distribution shifts in generalization, and how
About me
Hi! I’m Ying. I am currently a Wojcicki-Troper Postdoctoral Fellow at Harvard Data Science Initiative, where I have the fortune to be mentored by Professor José Zubizarreta and also work with Professor Marinka Zitnik.
Starting July 2025, I will be an Assistant Professor in the Department of Statistics and Data Science at the Wharton School, University of Pennsylvania. I obtained my PhD in Statistics from Stanford University in 2024, advised by Professors Emmanuel Candès and Dominik Rothenhäusler. Prior to that, I studied Mathematics at Tsinghua University. Here are my CV, Github and Google scholar pages.
I currently help organize the Online Causal Inference Seminar.
Research interests
I work on statistical problems related to two main themes:
- Uncertainty quantification
I develop methods to quantify and control the uncertainty of AI models, aiming for their principled use in high-stakes domains like drug discovery, medical AI, and automated scientific discovery. One focus is conformal selection, which finds unlabeled instances with “good” labels—such as active molecules and trustworthy LLM outputs—that can be acted upon with confidence. Recently, I have also been exploring statistical inference in automated scientific discovery with AI agents.
- Generalizability and robustness
I am interested in the generalization and robustness of statistical findings across datasets, populations, and contexts. My recent works study the empirical patterns of distribution shifts in large-scale replication studies. I also develop methods that address distribution shifts in generalizing treatment effects, learning causal decision rules, and combining datasets.
These questions lead me to the fields of conformal prediction, causal inference, and multiple testing.
News
- May 2025: I’m organizing an invited session on generalizability, transportability, and distribution shift at ACIC 2025!
- Apr 2025: I gave a talk on our POPPER agent framework at the International Seminar on Selective Inference! [slides] [recording]
- Feb 2025: Imagine LLM agents for scientific discovery—agents that autonomously gather knowledge by creative reasoning and flexible tool use. How to ensure the soundness of what they acquire? We propose POPPER, a framework where LLM agents design sequential experiments, collect data, and accumulate statistical evidence to validate a free-form hypothesis with error control!
- Dec 2024: Recent empirical investigations have challenged the sufficiency of covariate shift adjustment for generalization under distribution shift. How to address what remained unexplained? Analyzing two large-scale multi-site replication projects, our new paper suggests a predictive role of covariate shift: it informs the strength of unknown conditional shift, which helps generalization!
- Nov 2024: Excited to share Optimized Conformal Selection for optimizing/selecting conformity scores to pick as many “interesting” instances as possible (e.g., active drugs, trustable LLM outputs) while maintaining FDR control – even without sample splitting!
- Sept 2024: Outputs from black-box foundation models must align with human values before use. For example, can we ensure only human-quality AI-generated medical reports are deferred to doctors? Our paper Conformal Alignment is accepted to NeurIPS 2024!
- Sept 2024: I’m thrilled to receive the 2025 IMS Lawrence D. Brown PhD student award!
- Sept 2024: My paper on optimal variance reduction in online experiments (2021 internship project at LinkedIn) receives the 2024 Jack Youden Prize for the best expository paper in Technometrics! Thank you, ASQ/ASA!
- March 2024: How to quantify the uncertainty for an “interesting” unit picked by a complicated, data-driven process? Check out JOMI, our framework for conformal prediction with selection conditional coverage!
- Sept 2023: I’ll be giving a seminar at Genentech on leveraging Conformal Selection [1, 2] for reliable AI-assisted drug discovery.
- Sept 2023: Scientists often refer to distribution shifts when effects from two studies differ, e.g. in replicability failure. Do they really contribute? See our preprint for a formal diagnosis framework. Play with our live app, or explore our data repository! I gave an invited talk about it in the Causality in Practice Conference.
Beyond academics, I love traveling and photography in my free time. See my photography gallery!
Education
- Ph.D, Statistics, Stanford University, 2019 - 2024.
- B.S. Mathematics, Tsinghua University, 2015 - 2019.
- B.A. Ecomonics (Finance), Tsinghua University, 2015 - 2019.